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THE HILLEGAS-THORNDIKE SCALE FOR MEASURE- 
MENT OF QUALITY IN ENGLISH COMPOSITION 
BY YOUNG PEOPLE 



FRANKLIN W. JOHNSON 
Principal University High School 



A scale for measuring the merit of English composition by 
young people, prepared by Professors Hillegas and Thorndike of 
Teachers College, is given in Professor Thorndike's recent book, 
Education. 1 The method employed in deriving the scale is given 
in detail in a recent number of the Teachers College Record.' Some 
seven thousand compositions of young people were divided into 
ten classes, each roughly representing the same degree of excellence. 
From these classes seventy-five samples were selected. To these 
were added artificial samples of poorer quality than could be found 
in actual school work, in order to fill in the lower ranges of the 
proposed scale. All together the set consisted of eighty-three 
samples which varied from the poorest to the best by small degrees 
of quality. About one hundred individuals arranged these samples 
in the order of their excellence. From the judgments of these 
individuals a smaller group of twenty-seven samples was selected 
as containing all the important steps in quality from the poorest 
to the best. Two other samples were added as the investigation 
proceeded, to fill in gaps in the scale. Five hundred and two 
individuals judged these sets of samples and on the basis of their 
judgments the scale was derived by the method of right and 
wrong cases. 3 The theory of this method as applied to this study 
is stated as follows: "Differences that are equally often noticed 
are equal, unless the differences are either always or never noticed." 
The zero point in the scale was located by the combined judgment 
of twenty-eight individuals, nine of whom were "men of special 

* Edward L. Thorndike, Education, The Macmillan Co., pp. 214-19. 
a Milo B. Hillegas, Ph.D., "A Scale for the Measurement of Quality in English 
Composition by Young People," Teachers College Record, September, 1012, pp. 1-54. 
3 Fullerton and Cattel, On the Perception of Small Differences, p. 12. 
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literary ability," eleven "gifted teachers familiar with secondary 
education," and eight "psychologists familiar with the significance 
of scales and zero points in the case of intellectual abilities and 
products." The scale as finally formed represents the combined 
judgments of five hundred and fifteen individuals who read in 
whole or in part the material from which the scale was finally 
selected. The scale is as follows: 



Dear Sir: I write to say that it aint a square deal Schools is I say they 
is I went to a school, red and gree green and brown aint it hito bit I say he 
don't know his business not today nor yeaterday and you know it and I want 
Jennie to get me out. 

18 

the book I refer to read is Ichabod Crane, it is an grate book and I like 
to rede it. Ichabod Crame was a man and a man wrote a book and it is called 
Ichabod Crane i like it because the man called it ichabod crane when I read 
it for it is such a great book. 

26 

Advantage evils are things of tyrrany and there are many advantage 
evils One thing is that when they opress the people they suffer awful I think 
it is a terrible thing when they say that you can be hanged down or trodden 
down without mercy and the tyrrany does what they want there was tyrans 
in the revolutionary war and so they throwed off the yok. 

37. Sulla as a Tyrant 

When Sulla came back from his conquest Marius had put himself consul 
so sulla with the army he had with him in his conquest seized the government 
from Marius and put himself in consul and had a list of his enemys printy and 
the men whoes names were on this list we beheaded. 

47. De Qutncy 

First: De Quincys mother was a beautiful women and through her De- 
Quincy inhereted much of his genius. 

His running away from school enfluenced him much as he roamed through 
the woods, valleys and his mind became very meditative. 

The greatest enfluence of De Quincy's life was the opium habit. If it 
was not for this habit it is doubtful whether we would now be reading his 
writings. 

His companions during his college course and even before that time were 
great enfluences. The surroundings of De Quincy were enfluences. Not 
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only De Quincy's habit of opium but other habits which were peculiar to 
his life. 

His marriage to the woman which he did not especially care for. 

The many well educated and noteworthy friends of De Quincy. 

58. Fluellen 

The passages given show the following characteristic of Fluellen: his 
inclination to brag, his professed knowledge of History, his complaining 
character, his great patriotism, pride of his leader, admired honesty, revenge- 
ful, love of fun and punishment of those who deserve it. 

67. Ichabod Crane 

Ichabod Crane was a schoolmaster in a place called Sleepy Hollow. He 
was tall and slim with broad shoulders, long arms that dangled far below his 
coat sleeves. His feet looked as if they might easily have been used for shovels. 
His nose was long and his entire frame was most loosely hung together. 

77. Going Down with Victory 

As we road down Lombard Street, we saw flags waving from nearly every 
window. I surely felt proud that day to be the driver of the gaily decorated 
coach. Again and again we were cheered as we drove slowly to the postmasters, 
to await the coming of his majestie's mail. There wasn't one of the gaily 
bedecked coaches that could have compared with ours, in my estimation. 
So with waving flags and fluttering hearts we waited for the coming of the 
mail and the expected tidings of victory. 

When at last it did arrive the postmaster began to quickly sort the bundles, 
we waited anxiously. Immediately upon receiving our bundles, I lashed the 
horses and they responded with a jump. Out into the country we drove at 
reckless speed — everywhere spreading like wildfire the news, "Victory"! 
The exileration that we all felt was shared with the horses. Up and down 
grade and over bridges, we drove at breakneck speed and spreading the news 
at every hamlet with that one cry "Victory"! When at last we were back 
home again, it was with the hope that we should have another ride some day 
with "Victory". 

83. Venus of Melos 

In looking at this statue we think, not of wisdom, or power, or force, but 
just of beauty. She stands resting the weight of her body on one foot, and 
advancing the other (left) with knee bent. The posture causes the figure to 
sway slightly to one side, describing a fine curved line. The lower limbs are 
draped but the upper part of the body is uncovered. (The unfortunate loss 
of the statute's arms prevents a positive knowledge of its original attitude.) 
The eyes are partly closed, having something of a dreamy langour. The 
nost is perfectly cut, the mouth and chin are moulded in adorable curves. 



42 THE SCHOOL REVIEW 

Yet to say that every feature is of faultless perfection is but cold praise. No 
analysis can convey the sense of her peerless beauty. 

93. A Foreigner's Tribute to Joan of Arc 

Joan of Arc, worn out by the suffering that was thrust upon her, never- 
theless appeared with a brave mien before the Bishop of Beauvais. She 
knew, had always known that she must die when her mission was fulfilled 
and death held no terrors for her. To all the bishop's questions she answered 
firmly and without hesitation. The bishop failed to confuse her and at last 
condemned her to death for heresy, bidding her recant if she would live. 
She refused and was lead to prison, from there to death. 

While the flames were writhing around her she bade the old bishop who 
stood by her to move away or he would be injured. Her last thought was 
of others and De Quincy says, that recant was no more in her mind than on 
her lips. She died as she lived, with a prayer on her lips and listening to the 
voices that had whispered to her so often. 

The heroism of Joan of Arc was wonderful. We do not know what form 
her great patriotism took or how far it really led her. She spoke of hearing 
voices and of seeing visions. We only know that she resolved to save her 
country, knowing though she did so, it would cost her her life. Yet she never 
hesitated. She was uneducated save for the lessons taught her by nature. 
Yet she led armies and crowned the dauphin, king of France. She was only 
a girl, yet she could silence a great bishop by words that came from her heart 
and from her faith. She was only a woman, yet she could die as bravely 
as any martyr who had gone before. 

The figures above each selection in the scale represent its 
exact value in units of the scale extending from o to 93. Quality 
37 ("Sulla as a Tyrant") is understood to differ from quality 47 
("De Quincy") by exactly the same amount as quality 67 
("Ichabod Crane") differs from quality 77 ("Going Down with 
Victory"); and quality 93 ("Venus of Melos") is understood to 
be just one point less than twice as good as quality 47. Professor 
Thorndike says: 1 "By using such scales, the absolute gain which 
any pupil made in any year could be measured in the same way 
as his gain in height, weight, wages, or pulse rate, and the results 
of different means and methods of teaching could be demonstrated 
with exactitude instead of being guessed at." 

There can be no doubt of the importance of securing definite 
standards for the measurement of school products in place of the 
indefinite and widely varying standards employed in different 

'Education, p. 214. 
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schools and by different teachers in the same school. Neither 
can there be any doubt regarding the elaborate pains with which 
this scale for the measurement of English composition has been 
derived. However, one who examines this scale can hardly fail 
at first to be skeptical regarding its practical availability for measur- 
ing the merit of so highly complex a product as the composition 
of young people in their teens; and those who are accustomed to 
scoff at the idea of a science of education will doubtless find material 
for ridicule in the claims made by its authors for the exactness 
with which this scale may be used. 

For the purpose of a practical test, the author of this article 
submitted eight selections taken from the actual compositions of 
high -school pupils to forty- two teachers of English composition 
in a number of high schools, normal schools, and colleges, all of 
sufficiently high standing to make their judgments worthy of 
respect. The selections were also submitted to two other groups 
of judges, one consisting of five teachers of English in the University 
High School, in which an effort has been made for some time to 
secure uniform standards of grading; the other consisting of a 
class of sixteen graduate students taking a course in the statistical 
study of school materials in the School of Education in the Uni- 
versity of Chicago. With the material to be graded was submitted 
to each a copy of the Hillegas-Thorndike scale and they were asked 
to grade the material carefully according to the scale and to sub- 
mit their results without consultation with each other. It is 
probably fair to assume that these individuals were as competent 
to apply the scale as the average persons who could be expected to 
apply it in actual practice. 

The material furnishing the basis of the test is given below, 
the different selections being designated by the letters A-H. It 
perhaps should be stated that selections A and B were written by 
members of an industrial class who had completed only six grades 
in an elementary school. 

A 

Football. The place was 50X100 ft, and there was a ten boys playing 
F. B. our captain was Kelley and the score was 26 to o. Its a very interesting 
game because it makes the man helty. 
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I like to play Football or baace Ball. But I dont know how to play 
eather one I like to learn it. 

B 

The boys call out at one time, we want to play football. One boy hollars 
captain on one side. The game is started, and I am afraid I will get hurth. 
All at once I see the boys running I run to I jump on one boy. I get a goo 
and tight grip, and I was down on the ground. 

C 

Anyone trying for this Rhodes scholarship must comply to three rules. 
The applicant must have had two years in some college of good standing must 
be unmarried, and must be between nineteen and twenty one years of age. 

"The rules seem quite easy for a scholarship, don't you think so girls?" 
asked Marion. 

"Yes, but then it is not so easy as you think because there are many 
other applicants," replied Helen, "But continue." 

Then, Mr. Porter told us about Oxford. There are twenty one colleges 
at Oxford, but they are shut in by a high wall and are separated from each 
other. 

"Why! I thought Oxford contained just one large college instead of 
twenty-one," said Helen. 

"Yes, I replied and there are a good many other things which greatly 

surprised me. 

D 

When one has a class on the third floor of Blaine Hall and one on the 
third floor of Belfield Hall the hour following, the five minutes allowed between 
classes does not make it possible for him to reach his other class on time. 
Now, when he is absent one day and stops the following day, after class to 
find out when he may make up his missed lesson he arrives at his next class 
late. Then perhaps, the teacher has some point she wishes to explain to 
him, and so retains him a moment. Maybe his locker combination will not 
work, or perhaps he drops his books, the result is the same in each case, he 
is late to class. He reaches the class room, five minutes late. He rushes in 
panting. For who wouldn't be breathless after running up three flights of 
stairs ? As he enters he is confronted with "why are you late ?" 

To which he pants out. "Third floor of Blaine, couldn't help it." 

E 
Of the many points which should be taken into consideration in choosing 
a college, one of the most important is that of the size of the institution. I 
am of the opinion that it should be rather small; that is, with an undergraduate 
body of about two hundred and fifty students. There are a number of reasons 
for this. In the first place, in a small college like this, a student who has 
exceptional powers in any branch of interests has more opportunity to come 
to the fore, and make the most of himself. For example, if a man is an unusu- 
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ally good runner, although he does not know it, and has not been well-developed, 
if he is in a small college, he is more likely to go out for track and find out 
what he can do, than if he is in a large university, discouraged by the great 
number of athletes around him who are, as this student thinks, so much better 
than he. Thus he does not try himself out, and some excellent material is 
wasted. If he had been in a smaller college, he would have tried himself 
out, and would have developed into a runner worth while, and would have done 
both himself and his Alma Mater good. On the other hand, if a man is excep- 
tionally bright, and good at his studies, if he is in a smaller college, he will 
have more opportunity for the individual attention of the instructors, than if 
he were in a large university, where, besides being distracted from his work by 
the multifarious interests of a large institution he is prevented from close 
contact with the professors who have to teach large classes and are hindered 
from much outside help of the students by the excessive burdens of adminis- 
trative work. 

F 
That night, we took dinner at the hotel and, as we were not over anxious 
to return to scene of destruction, a bunch of us stayed at the hotel the great 
part of the evening. When we finally did return, we talked and chattered 
freely as long as we were on the regular hotel walk, but when we turned off 
on our path through the woods, a sad silence descended upon us. Each of us 
felt a peculiar fear and gloom. As we came to the place where we turned, 
we stopped a few minutes, knowing that we would see no welcoming lights 
nor hear the pleasant noise of talking. When we turned, we saw the ruins 
bathed in a flood of most beautiful moonlight. The moonshine vied with the 
strange phosphorescent glow of the ruins, themselves, in making the scene 
wierd, and terrible. Two men kept watch through the long night, lest the 
fire break out, again. The next day, with sad hearts, we left the mournful 
ruins for the more cheerful, if less beloved, city of Chicago. 

G 

Down the trail to the beach we stumbled, our only illumination the little 
lamp, and went along the shore, very silently, for each one was too much 
occupied in shivering to say anything except muttered ejaculations when he 
stubbed his numbed toe against a rock. Soon, however, we came out on the 
road, where the footing was much smoother, and here we quickened our pace. 
After half an hour of pretty steady climbing, we emerged from the woods, and 
struck a little footpath, which led across the rocky pasture land where we 
really began to feel the cold. The grass was heavy with the first frost, and 
by the time we had gone a hundred yards, our legs from the knees down were 
thoroughly drenched with ice-water. But by this time, most of the journey 
was over, the light of dawn was crawling over the sky, the clear, cheerful call 
of a white-throated sparrow floated down from a fir nearby, and we really 
began to think that early morning was a pretty good time of day, after all. 
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H 

Here are the lockers where we kept our books. I wonder if I could open 
any of them. Here is the one I had during my last year in high school. I 
wonder who is its owner now. How does it look, in the interior. She and 
I kept our many belongings within, sometimes in good order, and sometimes, 
just before a wild and thorough cleaning, in a state of peculiar confusion. 
I did not like to clean it, then, but how I should enjoy going through it from 
top to bottom, now, and arranging the books, hers and mine, in neat rows! 
I try my old combination, but the locker will not open. Here is the other 
locker. There were four of us, in those days. The interior of this other 
locker used to be arranged much as ours. I wonder if the owners of these 
two lockers are as good friends as we four were. 

The following tables give the essential features of the results 
obtained from the gradings of the three groups, showing the lowest 
grade given to each selection by a member of each group, the highest 
grade, the maximum variation, the average grade, and the average 
variation of each individual from the average grade of the entire 
group. 

FORTY-TWO TEACHERS OF ENGLISH COMPOSITION IN HIGH 
SCHOOLS, COLLEGES, AND NORMAL SCHOOLS 



Specimen 


Lowest 
Grade 


Highest 
Grade 


Maximum 
Variation 


Average 
Grade 


Average Varia- 
tion from 
Average Grade 


A 


o 
o 
35 
47 
45 
40 
67 
26 


40 

47 
86 

85 
100 

93 

97 
95 


40 
47 
51 
38 
55 
53 
3° 
69 


17 
20 

53 
71 
85 
75 
84 
80 


8 


B 


8 


C 


11 


D 


8 


E 


8 


F 


8 


G 


7 


H 


11 







SIXTEEN MEMBERS OF GRADUATE CLASS IN EDUCATIONAL TESTS. 
SCHOOL OF EDUCATION 



Specimen 


Lowest 
Grade 


Highest 
Grade 


Maximum 
Variation 


Average 
Grade 


Average Varia- 
tion from 
Average Grade 


A 


15 
18 
26 
38 

32 
65 
67 
47 


47 
37 
83 
93 
95 
90 
94 
93 


32 
19 

57 
55 
63 
25 
27 
46 


28 
28 
59 
71 
74 
77 
81 
80 


II 


B 


6 


C 


15 

13 

8 


D 


E 


F 


6 


G 


7 


H 
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FIVE TEACHERS IN THE ENGLISH DEPARTMENT OF THE 
UNIVERSITY HIGH SCHOOL 
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Specimen 


Lowest 
Grade 


Highest 
Grade 


Maximum 
Variation 


Average 
Grade 


Average Varia- 
tion from 
Average Grade 


A 


12 
12 
47 
57 
75 
64 
77 
7S 


30 

42 
67 
80 
IOO 
87 
97 
95 


18 

3° 
20 

23 

25 

23 

20 
20 


19 
27 
60 
71 
85 
77 
86 
88 


5 

7 


B 


C 


II 


D 




E 


5 
5 
6 


F 


G 


H 


7 





From this application to school material at least two criticisms 
of the scale may be made: 

1. The scale represents the composite judgment of some five 
hundred individuals more or less expert as teachers of English, 
writers, and experts in fields that render their individual judgments 
worthy of respect. But the scale represents the judgment of no 
single individual, and the judgment of any single individual or 
of any group of individuals whose judgment entered into the 
formation of the scale such as might reasonably be expected to 
apply the scale to any large body of material necessary to an 
adequate practical test would not represent the aggregate judg- 
ment of those whose arrangement of the material formed the basis 
of the scale. These individuals varied widely in their judgments, 
placing the various selections in rather widely varying positions 
in the scale. But the scale cannot be used in practice by all the 
persons whose judgments contributed to its formation. It must 
be used by an individual or by a small group of individuals whose 
judgments may be expected to vary as widely as did those in the 
making of the scale. As there is no such person as the average 
pupil, so there is no average judgment which can be applied to a 
test of school products. This of course could be said of any 
similar test to be applied to school products and furnishes in itself 
no adequate reason for the rejection of this scale. It is, however, 
a more serious objection in the case of the composition scale 
because of the highly complex nature of the material to which 
it must be applied. 
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2. A more serious objection to this particular scale may be 
made on the ground that the material employed is not representa- 
tive. In estimating the value of written composition there are 
two elements that may roughly be said to be of about equal 
value — form and content. By form we would mean such elements 
as correctness or freedom from mistakes of spelling, punctuation, 
capitalization, syntax; neatness and general good appearance; 
effective and pleasing sentence structure; organization of material 
in logically coherent units. A scale for evaluating these elements 
might be applied with reasonable accuracy and for this purpose 
the present scale may perhaps be regarded as an excellent one. 
By content we would mean the quality of the thought, feeling, 
artistic skill, effectiveness, and originality of expression. As a 
basis for evaluating this important element, the scale seems to be 
entirely inadequate. Every selection in the scale that has any 
thought-content at all is on a subject exceedingly formal, drawn 
from history or most often from readings in literature. An excep- 
tion might be made of the scale number 83, "Venus of Melos," 
which may have been written by one who had seen a reproduction 
of the statue, but quite as likely by one who had read a description 
of the statue. One who reads the scale carefully might assume that 
for some reason all material involving any originality of thought 
had been carefully excluded. It cannot be supposed that composi- 
tion subjects in any school were chosen exclusively from such a bar- 
ren field as is represented by the material in this scale. Certainly 
no school should be expected to furnish any large amount of 
material on subjects so far removed from all present-day human 
experience. It is altogether impossible to compare compositions 
on subjects that offer opportunity for originality of thought and 
expression with the formal material found in the scale which depends 
for its content largely upon the memory of books read or discussed 
in class. The scale differs in kind from the material to be measured. 
It is like using a yardstick to determine the weight of material in 
the physical laboratory. 

Good illustrations of this essential difference between the 
scale and the material graded may be found by a comparison of a 
few of the selections graded with that part of the scale to which 
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the average grade assigned makes them most nearly equal. Selec- 
tion B is given an average quality 18 in the scale. While very 
crude in form, it will be observed to have a certain vividness and 
unity of form which the corresponding selection in the scale does 
not possess. The wide variations in the grading of this selection 
are doubtless due to the difficulty which the judges experienced 
in making an exact comparison of material so essentially different 
in content. The most marked variation occurred in the grading 
of selection H, in which some judges distinguished a certain fine 
feeling which influenced the style and gave it unity while one who 
gave it a very low grade evidently did not observe this excellence 
at all. From this inherent difficulty in measuring two such dis- 
similar elements as form and content in written composition, one 
might suggest the desirability of two scales for the determination 
of the value of these two elements. 



